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Research Report fflhU 
Jack H. Bond 
March 197^ 

A PREVIEW OF RECENT FINDINGS 
ABOUT EMH STUDENTS TO FILMS 

ABSTRACT 

The model developed by the Computer Based Project in Syracuse, 
Y. to evaluate the use of captioned filras for the deaf with mental- 
ly handicapped and emotionally disturbed children is briefly described, 
followed by a review of recent research conducted by the Project Staff* 
Among the areas which have been researched ares (l) pretest-posttest 
design modification, (2) effects of confirming correct answers,; (3) 
paper and pencil vs. automated response wodes, (h) alternatives to the 
multiple choice test format, (5) non-verbal assessment techniques, 
(6) attention Variables (e* g« colo^p subject matter, etc.)| and (7) 
captioning effects. 
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SPECIAL REPORT No. 
COMPUTER-BASED PROJECT for the, EVALUAT I ON 
of MEDIA for ths HANDICAPPED. 

Title. A PREVIEW OF RECENT FINDINGS ABOUT BMH STUDENTS TO FILMS 
BACKGROU ND 

The Computer* Based Project for the Evaluation of Media for the Handicapped, 
based on contract #OEC-9-423617-4357 (616) between the Syracuse (N.Y.) City School 
District and the Media Servi9es and Captioned Films Branch, Bureau of Education 
for the Handicapped (United States Office of Education) for the five year period 
July 1, 1969 through June 30, 1974. The major goal is to improve the instruction 
of handicapped children through the development and use of an evaluation system 
to measure the instructional effectiveness of films and other materials with 
educable mentally handicapped (EMH) children, in-service training and media support 
for spocial teachers, and studies related to the evaluation process and the 
populations used. 

The Project has concentrated on the 600 films and 200 filmstrips from the- 
Media Services and Captioned Films (BEH - USOE) depository; however, specific 
packagos from Project LIFE, various elementary math curricula, and selected 
programs from Children's TV Vtorkshop have al^o been evaluated. The evaluation 
model used requires thati 1) objectives of materials be specified and written; 

2) instriunents be constructed to test and measure ef fectiveness; andp 3) children 
be the major sources of evaluation inform.ation. A number of instruments and 
methodologies are employed in the gathering of c^gi Itive and affective data from 
900 EMH chi3dron and 80 special teachers to make we effectiveness decisions - 
Over half of the EMH population can neither road or write; therefore, a unique 
Student Response System (SRS) is employed, consisting "of a twenty station G.E.- 
1000 SRS which can be operated in a group or individual recording mode and is 
connected to a remote computer system. The computer capabilities consist of 
remote telephone connections to the Rome (N.Y,) Air Development Command, the 
Honeywell time-shared network, and the Schenectady (N,Y.) G E Research and 
Development Center; and batch mode capabilities of the Syracuse City Schools, 
Syracuse University, and various corr^mercial sources. 

In-service and media support activities provide on-the-job training for 
teachers, teacher aides, equipment, and materials to the special teachers in 
the city schools. The research activities have centered around invr-stigations 
and special problems related to the development of the evaluation model. The 
four major areas considered are: 1) testing effects, 2) captioning effects, 

3) special student characteristics; and, 4) evaluation procedures validation. 

Docuraentation of the major activities appear in the five annual reports 
and the 600 evaluations prepared on materials used. Staff members v;ere encouraged 
to prepare special reports and the attached paper is one of those. The opinions 
expressed in this publication do not necessarily reflect the position or policy 
of the Computer Based Project, the United States Office of Education, or the 
Syracuse City School District, and no official endorsement by any of the agencies 
should be inferred. 



Research Report #7^*^ 
Jack H, Bond 
March 19lk 

A PREVIEW OP RECENT FINDINGS 
ABOUT EMH STUDENTS TO FILMS 

The Computer Based Project for the Evaluation of Media for the 
Handicapped was funded by the Bureau of Education for the Handicapped 
to develop a process to determine the effectiveness of 600 captioned 
films, intended for deaf students, on educable mentally handicapped 
(EMH) children. In the accomplishment of this assignment, some 80 
investigations were carried out to verify assumptions and test tech- 
niques being employed on the EMH population and on the films and film- 
strips provided from the Media Services and Captioned Film Branch 
depository. 

The Syracuse City Schools housed the Project in two large rooms 
in the basement of an inner-city school. Here a facility was installed 
to store the materials, provide staff space and a projection room for 
15 students. In this room a student response system (SRS) was Installed 
to automate the data collection into a remote telephone line to Rome Air 
Development Command computers. The City schools had a population of 900 
EMH children from six to sixteen years of age located in 80 teaching 
stations across the city. Fourteen of these were located in the same 
building as the project facility. The others were scattered In ten other 
buildings and were either transported to the center or were shown the 
media in the classroom, 

The purpose of this paper is to highlight recent findings particu- 
larly those dealing with color and bUck-white films and with captioned 
and non-captioned m^<^i 



In order to explain these studies I want to first point out the 
bases for the assumptions made. The design of the evaluation process 
Is a simple pretest-treatment-posttest design. At first this was 
rather strictly followed using the same children. Later it was modi- 
fied to use independent samples for the pretest and posttest data when 
it was determined that the pretest lengthened the required attention 
span and consequently lowered the posttest scores. 

Spaid and Plotnick (1973) found a significant gain in posttest 
score with EMH subjects in a filmstrip treatment sequence. Two pre- 
tests were administered prior to the viewing of the filmstrip. This 
design would allow the determination of any test effects that might 
result by the pretest. No significant contributions were found from 
the pretests, but significant changes were found after viewing the film- 
strip. These changes (gains) have been consistently verified In every 
investigation where pretest and posttest scores are compared. V 

Comparisons of data collection techniques have been involved in 
several studies. Anastasio (1972) found no significant differences 
between confirming the correct answer after the group had responded in 
the SRS^and not confirming the answers in terms of improving the number 
of correct scores. However, the confirmation provided an observable 
motivation response in the fom of audible cheers and seemed to sustain 
the interest in answering the items. 

Bond (1971) found no significant differences In the number of 
correct answers between the scores from paper and pencil responses and 
those from the SRS. Again the motivation provided by giving confirm- 
ation after each response seemed to highlight the activity at a greater 



level than the paper and pencil mode. In another study Bond and 
Flowers (1972) set up a situation in the SRS where a ''2" response was 
the correct answer for every question in order to test the use of 
answer patterns by EMH children. After seven items only twenty per- 
cent of the population seemed to discover the ''secret even with con- 
firmation after each item, that "2" was the correct response. 

Alternate techniques to the multiple-choice itein were also at- 
tempted* Using open-ended questions in personal interviews, it was 
ascertained that there was very little difference between the younger 
(primary) and older (secondary) EMH students. Host replies were short 
answers, often employing a simple yes or no. The setting, sequence of 
events, and simple cause and effect relationship could be identified 
but not necessarily verbalized. The specific film had implied father- 
daughter, boy-girl, and peer relationships and these were not identl-* 
fiedi It was concluded that the low verbal level observed In these 
EMH children would make the employment of semantic scales or different 
tials impossible. 

A photo sequencing task was tested by Anastasio (1973) in which 
the EMH student was asked to place a series of photos in the order 
representing the concepts presented in lieu of answering questions 
about the concept. He found non-significant but positive correlations 
between Judged verbal and visual abilities and the correct sequenced 
photos, There was a significant difference between the scores of those 
who had seen the film and those who had not. Another non-verbal tech- 
nique Involved the observation of hand position on the body based on 



Body Image theories (Freud, 1927; Fisher, 1958). Tvo films were selec- 
tedi one had a Judged high affective message whereas the other had a 
heavy factual content. It was found that the hands were observed to 
he placed on the body and legs for the first filia at a much higher fre- 
quency than on the second film. The second had a much higher incidence 
of head-and external-placement and almost no body touching. Body 
image theories suggest the first as "protective" or more affective and 
the latter as "cognitive" or thinking activity. The investigation 
demonstrated that differential findings are observable from non-verbal 
responses and suggests that interpretations of the responses could be 

logically assigned to them. 

The foregoing summary has indicated that EMH children do give in- 
formation and exhibit behaviors which could be interpr^tMM^ related 
'to the mediated experiences to which they are being subjected. What 

are some of these relationships? 

of over 20.000 responses to cognitive items presented 
af«er viewing films has indicated that achievement variables on reading 
arithmetic, and spelling measures and IQ scores account for only six 
percent of the variance. Arithmetic scores were the highest predictors 
accounting for foui' percent, and full scale IQ accounts for one per- . 
cent. Budoff (1971) found similar results when a sample of nine films 
wer^ found to correlate positively with the series subtest on the 
Learning Potential instrument he was developing. The posttest scores 
on six of the nine filins were significantly correlated with children's 
achievement scores (p«.05). « 



The posttest scores for films (not separating color and black- 
white) were highest for health and language arts films and lowest for 
fine arts films. Higher scores were generally made on color films by 
five percent (when not separated by subject matter) and the difference 
was not significant. In terms of gain scores, the highest gains were 
for science and social studies and lowest for guidance and health 
films. Black-white films had generally higher gain than color films, 
with only higher posttest scores in fine arts. Black-white seemed to 
have lower pretests, again not significantly different. 

Posttest Attn Rank 
Longuage 65.8 88.1 3 

Health 63:6 70-0 I 

Guidance 61 • I qq^q 2 ' 

$00 Studies 56.7 89 .9 M 

Moth 55:6 76.0 7 

Science 55/2 83.2 5 

Bine Arts 52,3 83.0 6 

These posttest scores were related to an attention measure* Vfhen 
ranked by content area, the top four subject areas by posttest are also 
the top four in attention. These are health, social studies, language 
arts and guidance. Lowest attention was for math films » The younger^ 

students had higher attention fov color films and the older students 
attended more to black-white films. The cognitive content of the^^^^^^^ ^^^^^^^^^^ : 
films shown to each level may be an intervening variable • 



• PRIMARY 



Col.or 
86.8 



B-W 
8H.7 



INTERMEDIATE 86.0 87.4 
SECONDARY gS.H 88.9 

Mean Attent i on 

The attention measurement is made by observing five students as 
they viev a film and record on a five second interval the frequency 
of those viewing. This string of observations is summarized into a 
time series profile. 

The two hundred films available in the data base (20,000 records) 
have mean attention which varies usually between 75 and 85 percent 6f 
the children watching all the time. This results in a relatively 
small variance located at the upper end of the scale. Repeated obser- 
vations tend to give a similar curve 3md rather consistent mean values. 



Eight Prof i I es 



Between Groups M 209.7 72.M .31 

Residual , 35 8086.2 231.0 

Totol 39 8375.8 

F i ve •Prof i I es 



c/P s«i - MS 



Between Groups 7 983.8 140.5 2.06 
Residual 32 2183.9 68.2 

Total 39 3167.8 



For instance a study of five films with eight separate profiles pro- 
duced no significant deviations. Another study of six films with five 
profiles each produced a significant difference with one film "Rubber 
in Today's World," which was a rather heavy content film. Arranging 
the 200 mean attention index into a quartile scale further points out 
the small range - .6 percent in 2^-k9i 10 percent in 50-7J+; and 89 
percent in 75-100. 
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The profile of the bar graphs for posttest scores and mean atten- 
tion is almost the same when using a breakdown by content area. 
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AVERASE ATTENTION 



Tl\e first two variables singled out to he reported here are color 
and black-whi€e films. From the 600 films available, those with pre- 
test and posttest data were selected and processed with a one way 
ANOVA to detect differences between the tests, Sixty-six films re- 
sulted in a mean pretest of 38»8 percent correct &nd a mean posttest 
of 51 percent correct with a standard deviation of 10 for both, A 



One-Way ANOVA 



of ' SS /v)s 



Color/ BW I 1071 .43 1071 -43 7.45* 

Residual 82 I 1778.38 143.89 

Total 83 12867.81 



Gr i t ico l vol ue 7,95 (jf,j ^5q=^4.00 



s ignificant F value of f \k 5 was obtained for p«.0 5, Twenty-one films' 
of each type (color or black-white) were selected sequenclally as they 
appeared on a listing, and the pretest and posttest scores were ob- 
tained* A two-way analysis of variance was used to summarize the data* 



Two-way ANOVA 


S 


MS 


P 


Pre-Post 772. 10 


1 


772. 10 




Color/ BW 52.70 


1 


52.70 


.59 


Interoctlon HO. 00 


1 


HO. 00 


.H4 


Error 7208/00 


80 


70.13 




Totol 8073.00 


83 















The resulting analysis indicated no significant differences "be- 
tween color and black-white or for interactions between tests and color. 
Pretest and posttest differences were significant, F=8,8, 

' These findings are essentially the same as was foxind from the 
.analysis of the 20,000 responses; The values of the^ sample are reversed 
from that of the total sample. Color scored higher on posttest than 
did black-white in the lar^^er population, and black-white gained more. 
But these differences could be attributed to sample variation. 

The second two variables are captioned and non-captioned effects. 
Two studies were conducted to isolate the variables in two different 
measurements - cognitive differences and preference or attention dif-. 
f erences . 

The first study was conducted using a captioned and non-captioned 
version of the same film "1536-Magnets and Their Uses." A comparison 
Of the script of the audio sound track and the captioning revealed no 
essential differences in the number of words used or their rated dif- . 
ficulty by reading level. Two EMH classrooms at each primary and in- 
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termediate ley^el, were randomly assigned to view either the captioned 
or noH'^captioned version which were shown in adjacent classrooms 
simultaneously. Following the film showing the children moved back 
to their own room and were administered a cognitive test of eleven items 
on magnets. The attention profiles were token during the showings. 





S S 


M S 


df 


F 


Copt ion 


301 .77 


30( .77 


1 


2.30 


Leve 1 s 


22480.07 


22M80.07 


1 


171 .24 


1 nteroct ion 


171 .50 


171 .50 


1 


1 .31 


Error 


6826.57 


131 .28 


52 




Total 


29779.73 




55 





^Tobled Fl^L = 4.043 



The results were summarized in a two-way analysis of variance using- 
captioning and non-captioning as one set of variables and primary and 
intermediate as the other set . No significant differences were fouled 
6n the cognitive test between captions and levels. A significant 
(p=.05) difference was found (F=171.0) between the levels, with the 
intermediate children doing much better on the test than the primary 
children. Significant differences (using Chi-square) were also noted' 
between the attention profiles of the captioned and non-captioned' 
Showings. The non-captioned groups had a higher profile than did the 



captioned groups. Note the similarities in the trace of the two pro- 
files mentioned earlier in this reviewi 




1556 Magnets I ndependent Show! ng 

A second study to determine preference to the captioned or non- 
captioned version was organized where both versions of the film were 
shown in the same room simultaneously . The attention observation 




was modified to add the dimension of which screen was being watched* 
The resulting attention profiles were summarized and graphed for each 
film. Here we see the greater attention to the non-captioned version , 
found in the first study, was not verified. Subjective comments of the 
three observers verified the fact that the child seemed to watch the 
screen closest to him rather than to indicate any preference. The 
novelty of having two stimuli in a room may have also confounded the 



Non-copt ioned-^ 




profile. There may have also been a systematic variable working in 
study I to give the differences in profile; however, this Is uinlikely^ 
The cognitive test responses were essentially the same as in study I 
with a mean of 29 for primary and 65 for intermediate level • 

The findings of the differences between color and black-white films 
seems to suggest that the dimension of color does not significantly en- * 
hance the learning of cognitive information. The addition of captions 
does not seem to enhance learning in EMH children; in fact, it may 
even be suggested that the addition of captions may cause less atten- 
tion to be paid to the presentation* 

The generally higher cost of color and captioning does not seem 
Justified on the basis of academic learning for H4H children. The 
aesthetic and attention values, particularly for younger EMH children, 
may be important concerns to consider in order to Justify the added 
expense. 
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